Network storage system is based on the storage network, and storage server is the core of the network. This system provides data storage services for the application servers. The system emphasizes data services scalability, reliability and ability to handle dynamically. The most commonly used technology is the Storage Area Network (SAN). Storage area network is an independent network made up of storage servers, storage devices and storage network equipment. The storage area network provides a high-speed, dedicated, centralized managed, and security architecture.
In SAN, with the adding of storage servers, the overall I/O performance drops, therefore the shortcoming limits the use of the storage architecture. Particularly, with the development of the independent metadata server architecture, the dynamic scalability of storage servers (such as metadata servers) has become even more crucial issue to overall system performance. In the design of current metadata server, metadata servers usually use their own storage space to store metadata. The own storage space here means the storage space from the server's own hard disks. In general, metadata server cluster (storage server cluster), mainly uses two methods of data storage and management.
One method is that each metadata server of the cluster preserves part of the overall metadata table, so the overall metadata table is a combined table of all metadata servers' table. Under this architecture, each metadata server can achieve high data access performance and data storage efficiency without other synchro-mechanism. However, this design is bad during disaster and can not provide load balancing capabilities of metadata server cluster. The failure of a metadata server will lead to the loss of some metadata tables. The imbalance of metadata's dynamic load will lead to a lot of metadata access to one server, and the performance of metadata server cluster will drop.
The other method is that each metadata server of the cluster preserves a copy of synchronous overall metadata table in its local, so each metadata server can access the overall metadata table, including access of reading and writing. The metadata servers use a synchronization mechanism to ensure the integrity of overall metadata. So metadata server cluster can have a good balance of load capacity and disaster recovery capabilities. But usually metadata accesses are frequent small I/O requests, and the metadata server synchronization mechanism will seriously damage performance of the metadata server cluster, thereby the performance of the overall storage system will decline. Because of the cost of this synchronization mechanism, the metadata server cluster is limited in scalability. Because adding new metadata server means that we need to synchronize more overall metadata tables. This metadata management mechanism reduces the efficiency of metadata storage because each metadata record should have its corresponding local copy in every server.